Structural Data Parallel Neural Network Simulation

In this paper a novel approach for the data parallel simulation of neural networks on general purpose parallel machines is presented. We deene this new neural network simulation methodology algorithmically, and call it the 'Structural Data Parallel' approach. This approach is based on the Single-Program-Multiple-Data (SPMD) programming model and utilizes the highly specialized programming capabilities of high performance languages (like HPF, Vienna FORTRAN, etc.) for the parallelization process. The development process of the simulation system is reduced to the simple design of a sequential program, which is attributed with data distribution information. Thus the diicult task of physical parallelization is shifted to the programming environment and the compiler of the high performance language. An analysis of the runtime behavior of parallel neural network simulations developed according to this new scheme is presented, which justiies the eeciency of the new approach.